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The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE three MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36 (a). In no event, however, may a reply be timely filed after SIX (6) MONTHS from the 
mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even rf timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 ) 53 Responsive to communication(s) filed on Mar 27, 2003 



2a) 53 This action is FINAL. 2b) □ This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11; 453 O.G. 213. 

Disposition of Claims 

4) 53 Claim(s) 1-44 „ is/are pending in the application. 

is/are withdrawn from consideration. 

__ is/are allowed. 

6) K Claim(s) 1-24, 26, 28-30, and 32-44 ; is/are rejected. 

7) □ Claim(s) ' _ is/are objected to. 

8) D Claims 

Application Papers 

9) D The specification is objected to by the Examiner. 



4a) Of the above, claim(s) 25, 27, and 31 
5)D Claim(s) 



are subject to restriction and/or election requirement. 



10)D The drawing(s) filed on 



is/are a) □ accepted or b)D objected to by the Examiner. 



Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

11) D Jhe proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) D Acknowledgement is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some* c)D None of: 

1 . □ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. . 



3. □ Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
*See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgement is made of a claim for domestic priority under 35 U.S.C. § 1 19(e). 
a)D The translation of the foreign language provisional application has been received. 

15) D Acknowledgement is made of a claim for domestic priority under 35 U.S.C. §§120 and/or 121. 
Attachment(s) 

1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s>. 

2) □ Notice of Draftspersons Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-1 52) 

3) □ Information Disclosure Statement(s) (PTO-1 449) Paper No(s). 6) □ Other: 
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DETAILED ACTION 



Response to Amendment 

1 . In response to the Office Action mailed December 4, 2002 applicants have submitted an 
Amendment, filed March 27, 2003, amending claims 1 and 24 and adding new claims 41-44 
without adding new matter, canceling claims 25, 27, and 3 1 , and arguing to traverse claim 
rejections. 

Response to Arguments 

2. Applicants' arguments have been fully considered but they are not persuasive, for the 
following reasons: 

3. Applicants argue a difference between Rtischev's reading quality score and their 
intelligibility estimate, the latter said to measure "the degree to which others can understand a 
person's speech" (Amendment, p. 4). They assert that "a teacher's evaluation of a speaker's 
intelligibility is a subjective evaluation" while "comparing a transcript of what a listener hears the 
speaker say with the items that were spoken... provides an objective evaluation of a speaker's 
intelligibility" (p. 6). 

However, Rtischev et al. actually teach using a speech recognizer to determine a reader's 
speech quality, which measure they state is "preferably based" on "reading errors" determined by 
a "context-sensitive speech recognizer" (Abstract). By applicants' own argument, the resultant 
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speech-to-text transcription by the recognizer is then also a measure of the speaker's 
intelligibility. Moreover, a speech recognizer is even more "objective" than a listener who 
makes a transcription, because, unlike the recognizer, the listener would not necessarily produce 
the identical transcript when listening to the identical utterance on different days (as can be readily 
proved by having the same person transcribe the same, reasonably long, recorded speech). 

True, "A speaker may obtain a high reading quality score" from a speech recognizer (or a 
teacher) "even though a listener may have difficulty understanding the speaker". But, the reverse 
inconsistency can occur with a speech recognizer trained to different speakers versus a listener 
familiar with the speaker's accent, so that this has nothing to do with any supposed difference 
between speech quality and intelligibility. 

Despite applicants' argument that "The ability to read and the ability to be understood by 
others are two different abilities that require diffferent testing methods", the examiner still exists 
that Rtischev et al. and applicants are inescapably measuring both speech quality and intelligibility, 
for they go together, but their evaluations can vary with the background of the listener or the 
training procedure for the speech recognizer, in both cases depending on their respective degree 
of "tolerating strong foreign accents from a non-native user" (all quotes from Amendment, p. 4), 
which can vary. 

4. As for the newly-added limitations to claims 1 and 24 of "preparing a transcription of what 
heard" or "creating a transcription of what the listener hears", respectively, they are broad 
gh not to require the listener to be transcribing anything, for a speech recognizer located 



was 



enout 
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near a listener (or where a potential listener would be located) and doing speech-to-text 
transcription reads on these limitations. As indicated above, such a speech recognizer is taught by 
Rtischev et aL Also, their "script", which is being read for reading evaluation, also represents 
"what was heard" or "what the listener hears". 

5. Therefore, the amended pending claims stay rejected over Rtischev et al., and their 
rejection is repeated, mutatis mutandis for claim amendments and cancellations, below. 

6. As for the argument that co-pending application 09/31 1,617 does not involve 
obviousness-type double-patenting because "Townshend does not suggest or claim a method or 
apparatus for measuring intelligibility of a speaker using a transcription of what a listener hears", 
the examiner disagrees for reasons given above, indicating that "what a listener hears" is the same 
as what a speech recognizer determines. Thus, the provisional double-patenting rejection stands, 
and is repeated, mutatis mutandis, below. 

Claim Objections 

7. Claim 28 is now objected to because it still depends on canceled claim 27 . Appropriate 
correction is required. 



Claim Rejections - 35 USC 
8. The text of those sections of Title 35, U.S. Code not included in this action can be found 
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in the previous Office action. 

Claim Rejections - 35 USC §102 
9. Claims 1, 2, 5, 7, 8, 36, and 37 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Dimitry Rtischev et al (U.S. Patent 5,634,086, issued May 27, 1997). 

As per claims 1, 2, 5, 7, 36, and 37, Rtischev et a/.teach: 

• a means for hearing at least one person who is repeating items (spoken-language 
instruction apparatus employing speech recognition with user reading words from a 
written script from an inherent database, Abstract; user and microphone, or user and 
telephone, Fig. 1, elements 12 and 16, or elements 12 and 14, respectively); and 

• means for comparing the items with a transcription of what was heard and thus measuring 
intelligibility from the comparison (speech recognizer using nonlinear HMM speech 
models, Fig. 3, element 112; preselected script, element 114; score set, element 120; 
reading errors, col. 3, lines 43 and 47). 

As per claim 8, the "reading errors" (col. 3, lines 43 and 47) of Rtischev et al inherently 
include at least word substitutions, for an error in reading a word could cause the ASR to 
interpret it as a different existing word (e.g. a Japanese reader using Rtischev et a/.'s apparatus to 
learn English might pronounce "frame" as "flame", which would cause the ASR to recognize the 
spoken word as the latter). 
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Claim Rejections - 35 USC §103 



10. Claims 3, 4, 6, 8, 11-18, 24, 26, 28-30, 32, 33, 42, and 44 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Rtischev et al. as applied to claims 1, 2, 5, 7, 8, 36, and 37, 
above. 

As per claims 3, 4, 8, 1 1-13, 15-18, 24, 26, 28-30, 32, 33, 42, and 44, Rtischev et a/.do 
not teach a listener or a plurality of people capable of listening to the speaker. However, the 
examiner takes Official Notice that it is centuries old and notoriously well known to have teachers 
listen to speakers (students) so as to evaluate the intelligibility of their speech in terms of reading 
errors. Therefore, it would have been obvious for an artisan at the time of invention to have also 
people listening, at least during the training mode for the Automatic Speech Recognizer (cf. Fig. 
3, elements 102 and 104), to make sure the ASR does not make unreasonable "recognitions" or 
rejections. (Claim 28 has been reinterpreted as depending on claim 24.) 

The rest of the limitations were discussed in connection with the rejection of claims 
1, 2, 5, 7, 8, 36, and 37, above. 

As per claim 14, Rtischev et al. do not teach selecting listeners based on certain 
background characteristics. However, it would have been obvious for an artisan at the time of 
invention to select listeners that have extensive background speaking knowledge of the language 
being learned because they would be best able to determine the intelligibility of someone trying to 
speak the language. 
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11. Claims 9, 10, 19, 20-23, 25, 34, 35, 38-41, and 43 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Rtischev et al. as applied to claims 1, 2, 5, 7, 8, 36, and 37, above, in 
view of Charles Lewis et al (U.S. Patent 5,059,127, issued October 22, 1991). 

As per claims 9, 10, 19, 20-23, 25, 34, 35, 41, and 43, while Rtischev et al. teach 
evaluating an error count intelligibility score (reading errors, col. 3, lines 43 and 47), they do not 
evaluate difficulty of the items and ability of a listener, nor doing this using Item Response 
Theory. However, Lewis et al. do (col. 1, line 63 through col. 2, lines 1, 16-26, and 42-43, with 
Figure 1A). 

It would have been obvious for an artisan at the time of invention to do this because Lewis 
et al. teach that IRT "allows creation of a test in which different individuals receive different 
questions, yet can be scored on a common scale" as well as "permits determination in advance of 
test administration of the level of ability and the accuracy with which ability has been measured" 
(col. 2, lines 31-36). 

The rest of the limitations were discussed in connection with the rejection of claims 
1, 2, 5, 7, 8, 36, and 37, above. 



As per claim 38, Rtischev et al. do not teach a database containing data from previous 
evaluations. However, Lewis et al. teach retaining data from previous "testlets" and "sequentially 
administering testlets ... until a pass/fail decision can be made" (Abstract), thus suggesting 
retaining results of previous intelligibility evaluations (testlets) for later continued evaluation. It 
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would have been obvious for an artisan at the time of invention to do this, to avoid having to 
administer all the testlets in a single sitting. 

As per claim 39, Rtischev et al. teach evaluations using data selected from at least speaker 
responses and items (coL 3, lines 11-17). 

As per claim 40, Rtischev et al. suggest use of nonlinear artificial neural net models for 
speech recognition (see reference to Kim et al. under "Other Publications" on the front page, top 
of second column). 

Double Patenting 

12. The nonstatutory double patenting rejection is based on a judicially created doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or 
improper timewise extension of the "right to exclude" granted by a patent and to prevent possible 
harassment by multiple assignees. See In re Goodman, 1 1 F.3d 1046, 29 USPQ2d 2010 (Fed. 
Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 
F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 
(CCPA 1970);and, In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1 .321(c) may be used to 
overcome an actual or provisional rejection based on a nonstatutory double patenting ground 
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provided the conflicting application or patent is shown to be commonly owned with this 
application. See 37 CFR 1.130(b). 

Effective January 1, 1994, a registered attorney or agent of record may sign a terminal 
disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b). 

13. Claims 1-24, 26, 28-30, and 32-44 are provisionally rejected under the judicially created 
doctrine of obviousness-type double patenting as being unpatentable over claims 1-3, 5-9, and 11- 
15 of copending Application No. 09/31 1,617. Although the conflicting claims are not identical, 
they are not patentably distinct from each other because they claim obvious variations of the 
claims in the instant application. The operation of the speech recognizer constitutes preparing a 
transcript. 

This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in fact been patented. 

Conclusion 

14. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is 
reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
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the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

15. Any response to this action should be mailed to: 

Box AF 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

or FAXed to: 

(703) 872-93 14 (please label official communications 
"EXPEDITED PROCEDURE"; please label informal or 
draft communications "PROPOSED" or "DRAFT") 

Hand-delivered responses should be brought to Crystal Park 2, 2121 Crystal Drive, 
Arlington, VA, Sixth Floor (Receptionist). 

1 6. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to the examiner, Talivaldis Ivars Smits, whose telephone number is (703) 306- 
301 1 . The examiner can normally be reached Mondays-Fridays from 8:30 a.m. to 5:00 p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Marsha D. Banks-Harold, can be reached on (703) 305-4379. The facsimile phone number for 
Technology Center 2600 is (703) 872-9314. 
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Any inquiry of a general nature or relating to the status of this application should be 
directed to the Technology Center 2600 customer service, whose telephone number is (703) 306- 



0377. 




TAUWLWS AftRS SMJTS 
PRIMARY EXAMINER 



Art Unit 2654 
April 21, 2003 



